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Instrumentation rigor and bias management are major challenges for 
qualitative researchers employing interviewing as a data generation 
method in their studies. A usual procedure for testing the quality of an 
interview protocol and for identifying potential researcher biases is the 
pilot study in which investigators tty out their proposed methods to see if 
the planned procedures perform as envisioned by the researcher. 
Sometimes piloting is not practical or possible so an “interviewing the 
investigator ” technique can serve as a useful first step to create interview 
protocols that help to generate the information proposed and to assess 
potential researcher biases especially if the investigator has a strong 
affinity for the participants being studied or is a member of the population 
itself. Key Words: Interviewing, Instrumentation, Researcher Bias, and 
Qualitative Research 


Interviewing, along with field observations and document analysis, is one of the 
major ways qualitative researchers generate and collect data for their research studies 
(Gubrium & Holstein, 2003; Kvale & Brinkmann, 2008; Rubin & Rubin, 2006; Seidman, 
2006). Many of these researchers in the spirit of conducting naturalistic or discovery- 
oriented inquiries usually create study-specific questions for their interviews instead of 
utilizing pre-established questionnaires or survey instruments (Gubrium & Holstein). In 
such a manner investigators become the instruments through which data for their studies 
are collected or generated (Poggenpoel & Myburgh, 2003). 

Central to conducting research and more specifically qualitative research is the 
researcher as research instrument (Denzin & Lincoln, 2000: 368; Marshall & 
Rossman, 1995: 59-65). The researcher is the key person in obtaining data from 
respondents. It is through the researcher's facilitative interaction that a context is 
created where respondents share rich data regarding their experiences and life 
world. It is the researcher that facilitates the flow of communication, who 
identifies cues and it is the researcher that sets respondents at ease. This also 
contributes to a therapeutic effect for the respondents because they are listened to. 
(Poggenpoel & Myburgh, 2003, p. 418) 

When perfonning as a discovery-oriented research instrument, qualitative 
researchers tend to construct study-specific sets of questions that are open-ended in 
nature so the investigators provide openings through which interviewees can contribute 
their insiders’ perspectives with little or no limitations imposed by more closed-ended 
questions. The investigator would more likely use the more closed-ended style of 
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questions in a confirmatory type of study. In this mode of inquiry the investigators are 
attempting to confirm or test a hypothesis so it is important for the researchers to ask 
questions that allow them to confirm or disconfinn a particular notion. These closed- 
ended variety of queries typically start with verbs (e.g., Are you satisfied with the 
treatment provided by your doctor at your last office visit?) and tend to elicit narrowly 
defined answers or choices of responses (e.g., Yes, I was satisfied; or Check either: Yes, 
No, or Not Sure). 

In contrast investigators, who wish to discovery what is known about a particular 
phenomenon or situation from the insiders’ perspectives, tend to structure their interviews 
with open-ended questions which tend to start with words like who, what, where, when, 
why, and how (e.g., What was your experience of the treatment you received during your 
last office visit to see your doctor?) and suggest the respondent respond in a more 
expansive manner (e.g., I was really confused by what did and did not happen when I 
went to see my doctor for my yearly physical.). From the initial open-ended style of 
questions, curiosity-driven qualitative researchers then tend to employ follow up 
questions based upon the responses offered by the interviewee and designed to discover 
more details about the respondents’ particular experience (e.g., What made this last visit 
so confusing?) (Kvale & Brinkmann, 2008; Rubin & Rubin, 2006; Seidman, 2006). 

Developing the skills to conduct this style of interview takes considerable 
practice: 

Instrumentation is as critical in qualitative as in quantitative research. It takes 
training and practice to write open-ended questions, the hallmark of a qualitative 
interview, and then to keep from transfonning them into closed-ended questions, 
especially with a resistant subject, when actually conducting the interview. 
(Sofaer, 2002, p. 334) 

Despite its popularity, qualitative interviewing presents some challenges for 
researchers in terms of instrumentation rigor and bias management: 

The researcher as instrument can be the greatest threat to trustworthiness in 
qualitative research if time is not spend on preparation of the field, reflexivity of 
the researcher, the researcher staying humble and preferring to work in teams so 
that triangulation and peer evaluation can take place. (Poggenpoel & Myburgh, 
2003,p.320) 

These challenges may not only jeopardize the quality of a study, they may also 
prevent a study from ever being conducted if the protocol containing an open-ended 
interview fonnat cannot secure approval from an Institutional Review Board for the 
Protection of Human Subjects (IRB; Lincoln, 2005) due to questions of the approach’s 
ability to produce the results as suggested in the proposal. In other words, an IRB may 
question the integrity of the instrumentation if it has never been tested. If the instrument 
lacks rigor in the eyes of an IRB, they may not approve the protocol because the 
perceived weakness in the instrument can reduce their confidence that the investigator 
can conduct the study claimed in the protocol. In such a case the IRB can make the 
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determination that this deficit in the method reduces the benefits of the proposed study 
and thus may produce an unfavorable benefits-risk ratio to the participants. 

The role the researcher serves in this instrumentation also raises concerns 
regarding bias (e.g., Mehra, 2002). Poggenpoel and Myburgh (2003) suggest the potential 
reasons for this bias can include (a) the researcher’s mental and other discomfort could 
pose a threat to the truth value of data obtained and infonnation obtained from data 
analyses; (b) the researcher not being sufficiently prepared to conduct the field research; 
and (c) the researcher conducting inappropriate interviews (pp. 419-420). In addition to 
these reasons, the degree of affinity researchers have with the population under study 
including researchers being a member of the group themselves can introduce a question 
of bias in the study (Mehra, 2002). Given this affinity these “insider” investigators may 
limit their curiosities so they only discover what they think they don’t know, rather than 
opening up their inquiries to encompass also what they don’t know they don’t know. 

Pilot Studies 

A usual procedure for testing the quality of an interview protocol and for 
identifying potential researcher biases is the pilot study in which investigators try out 
their proposed methods to see if the planned procedures perform as envisioned by the 
researcher. 

The term pilot study is used in two different ways in social science research. It can 
refer to so-called feasibility studies which are "small scale version[s], or trial 
run[s], done in preparation for the major study" (Polit et ah, 2001: 467). However, 
a pilot study can also be the pre-testing or 'trying out' of a particular research 
instrument (Baker 1994: 182-3). One of the advantages of conducting a pilot 
study is that it might give advance warning about where the main research project 
could fail, where research protocols may not be followed, or whether proposed 
methods or instruments are inappropriate or too complicated, (van Teijlingen & 
Hundley, 2001, ^jl) 

In pilot studies investigators give their research method a “test run” by piloting 
their means for collecting and analyzing data on a small sample of participants with the 
same or similar inclusion criteria as would be the case in the main study. In this “dress 
rehearsal” researchers run through their study in an abbreviated form and make 
adjustments based upon the performance of the method. Data collected and analyzed 
during pilot studies are typically not included in the body of data generated in the main 
part of the study. 

A well-conducted pilot study can help investigators begin to address 
instrumentation and bias issues because they allow the researcher the opportunity to 

1. Administer the questions in the same way as in the main study 

2. Ask the subjects for feedback to identify ambiguities and difficult questions 

3. Record the time taken to complete the interview, decide whether it is reasonable, 
and better record participants’ time commitments in the IRB protocol 

4. Discard all unnecessary, difficult or ambiguous questions 
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5. Assess whether each question gives an adequate range of responses 

6. Establish that replies can be interpreted in terms of the infonnation that is 
required 

7. Check that all questions are answered 

8. Re-word or re-scale any questions that are not answered as expected 

9. Shorten, revise and, if possible, pilot again, (van Teijlingen & Hundley, 2001, ][6) 

Sometimes piloting is not practical because researchers do not want to lose 
limited research participants and their valuable information to a pilot study database not 
used in the study proper or the researcher does not want to take up participants’ valuable 
time with under-developed questions. Another problem with conducting a pilot study is 
IRB approval must still be secured if the piloting exercise will include human subjects. 
This situation will raise the instrumentation questions and risk-benefit issues as can occur 
with review process of the full study itself. 

To remedy these problems, investigators can turn to a pre-pilot study inquiry 
known as interviewing the investigator. In these cases this “interviewing the interviewer” 
technique can serve as a useful first step for investigators to create and revise interview 
protocols that can help address these IRB concerns, to generate the infonnation proposed, 
and to assess potential researcher biases especially if the researcher has a strong affinity 
for the participants being studied or is a member of the population itself (e.g., nurses 
studying nurses). 


Interviewing the Investigator 

In the interviewing the investigator approach the researcher assumes the role of a 
study participant and enlists a colleague to conduct the interview or the investigator can 
play both the role of the interviewer and interviewee. In both styles the interview is 
recorded and the researcher reviews the contents to see what infonnation was generated 
via the questions. If the research is being conducted as part of a thesis or dissertation, the 
investigators’ supervisor or chair can also participate as the interviewer or as an observer 
on the overall interviewing process. This oversight function can take place 
synchonistically as the interview is taking place or asynchronistically by reviewing a 
recording made of the interview. 

To start the interview of the investigator process, the interviewer would develop 
the series of open-ended questions that are planned to be used in the study. These should 
be written out and prepared in a fonn that will be used in the eventual interviews. For 
example, they may be word-processed and printed out on a piece of paper. If there are 
follow-up questions, these should also be noted. 

A space for conducting the interview should be selected that would resemble ones 
to be used in the study interviews and the equipment to be used in the eventual study 
should also be utilized. This step provides an added bonus of trying out the recording 
equipment to identify any technical problems that can arise. 

Before the interview ensues, the interviewer as interviewee should review the 
consent fonn and note any unclear or confusing passages and then sign the form before 
beginning the recording. Unlike a pilot study using “real” human subjects, the interviewer 
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can record this securing of consent interaction and note changes that can be made to 
improve the process too. 

Once consent has been secured, the interview can begin. When the process 
involves just interviewers interviewing themselves, they can first read the question and 
then answer the question. In the case of the two-party variation with another person 
serving as the interviewer, the process would unfold in a similar manner of asking 
questions and providing responses. The investigator can decide to run through the 
complete interview with or without interruptions or “time outs” to discuss or reflect on 
the proceedings. If inter-session breaks are used it is important to have a stop watch or 
some other such timing device so the time spent conducting the actual interview is noted. 
The interviewer can also use a second recorder to record these in-session discussions 
while stopping the original recorder so as to preserve a separation between the interview 
and the talk about the interview. 

Once the interviewing the investigator is completed, the participants can decide to 
try the process again and again until the results are satisfactory. Before these subsequent 
interviews are conducted the previous interview should be reviewed and critiqued. This 
process can be accomplished in a number of ways. The parties can play back the 
recording and cross-walk the questions as written out before the interview with the ones 
used and not used in the enacted interview. Notes can be made on what seemed to work 
and not work well in the interview. In examining the responses produced during the 
interviewing the investigator’s follow-up questions can also be noted and reactions to 
what was said and not said can be analyzed. Throughout this process moments of 
surprise, frustrations, and satisfactions should be articulated and assessed in relation to 
what was happening and not happening in the interview. 

As modifications are suggested or existing questions are confirmed the original 
list of questions should be edited and annotated. Investigators may wish to use editing 
tools found in software applications such as Microsoft Word to track changes being made 
and to insert comments to generate an audit trail in the question development and 
refinement process. 

In addition to these techniques researchers can also use methods such as 
journaling (Meloy, 1994) or interpersonal-process recall (IPR; Kagan, 1980) to examine 
thoughts and impressions that surfaced during the interview which might bias the 
collection and analysis of the “real” interviews of the study. In the journaling approach, 
researchers would record their thoughts before and after the interview. This could be 
done with a notebook or with a recorder. The process of writing/recording and 
reading/listening can help researchers identify heretofore unclear or unrecognized 
thoughts, feelings, and impressions which might have led to bias in the study if 
unchecked. 

With the IPR approach the interviewer and/or the interviewee listen to the 
interview again and stop the recording whenever a new question, impression, or 
observation arises. When such an occurrence happens, the recorded playing the original 
interview is stopped and a second recorder is started. The investigator makes a note of the 
place in the original tape that helped to initiate this observation and records the reactions 
that have arisen. In such a way the researcher can recall interpersonal and intrapersonal 
reactions from the process just encountered. This in-depth process can help to not only 
refine and improve the instrument, but can also help the researcher to identify potentially 
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biases as well as better identify with the cognitive and affective perspective of the 
research participant. This affinity can aid the researcher in developing empathy for the 
participants in the study and possibly identify some vulnerability or ethical concerns in 
asking certain questions. 

Once this cycle of interviewing the investigator process reaches a point of 
saturation (i.e., no new questions are being generated, no new modifications can be 
suggested, or no new potential biases can be identified; Guest, Bunce, & Johnson, (2006), 
the researchers can review the Instrument section of their IRB protocol and/or the data 
collection/generation section of the research proposal and make any adjustments based 
upon this quality control process. In editing these sections the researcher should clearly 
describe the interviewing the investigator process including how the determination of the 
integrity of the current interview format and question systems was decided. In the 
protocol/proposal the researcher can then chose to describe how a pilot study will also be 
used or how the study will move directly into the main stage of data collection. 

The data generated via the interviewing of the interviewer can also be analyzed 
just as the data from the participant interviews. This analysis can help researchers identify 
their pre-study thoughts and assumptions regarding what a participant might say in 
response to the questions being asked. The process of analyzing these responses can help 
researchers in their bracketing efforts (Seale, 1999) to manage subjectivity and the results 
can also be juxtaposed with the findings from the subsequent pilot and/or main study so 
researchers can note what results from the analysis of the participants’ reviews surprised 
them. Such a finding is always a good sign that the researchers were prepared to discover 
that which they did not anticipate rather than only discovering that which they could 
anticipate discovering. 


Discussion 

In addition to the ways in which a pilot study can help the researcher improve the 
data collection method, the interviewing the interviewer approach can also help the 
investigator to 

1. Identify personal feelings arising during the questioning 

2. Develop greater appreciation for the challenge of sharing all one knows about a 
topic 

3. Make overt perspectives that might bias the researcher in the study 

4. Leam the value of patience in the interviewing process 

5. Gain an appreciation of feelings of being and not being heard 

6. Appreciate the vulnerability of the participant 

7. Identify a priori assumptions about the participants 

Despite these advantages, the interviewing the interviewer also has its limitations. 
Researchers may remain blind to their biases, may be unable to anticipate problems with 
the study’s instrumentation, and may have unforeseen difficulties in utilizing the 
questions effectively. Of course these troubles can arise in studies that have used pilot 
studies too so the conjoint use of a pre-piloting technique like the interviewing the 
investigator approach with a pilot study may be recommended. In addition investigators 
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can use a system of peer debriefing (Seale, 1999) in concert with either approach to bring 
in different set of eyes and ears to help in the instrumentation improvement process. 

The advantages in using the interviewing the investigator remain in that rigorous 
testing of the instrument, both human and questions, can be accomplished without 
needing to secure IRB pennission or without having to remove valued participants from a 
limited resource due to the restrictions on using data from pilot interviews in the main 
study. The interviewing the investigator uniquely helps the researcher to assume the 
position of the research participate and gain a valuable perspective in what it feels like to 
be interviewed in the study. Such a point-of-view can prove to be invaluable in 
conducting research that is sensitive to the other in a study and can lead to more ethical 
and responsible research. 
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